Method of 3D Biometrics Identification

ABSTRACT

A method of 3D biometrics identification includes the steps of acquiring 3D image(s) of fingerprint; performing image normalization and feature extraction steps involving, noise elimination, ridge extraction, ridge modification and ridge validation; extracting 3D fingerprint minutiae and performing the reliability evaluation of the extracted 3D fingerprint minutiae; automatically extracting of 3D fingerprint template comprising at least five features; and automatically registering two 3D fingerprint templates, say P and Q, by computing relative 3D minutiae representation from a 3D minutiae in P (sample) considered as the origin and aligning every 3D minutiae in Q to compute best possible matching score, wherein every 3D minutiae in template P is further considered as the origin for the alignment of 3D minutiae in template Q, and the best possible matching score obtained from such considerations (candidate alignments) is considered as the final matching score between two 3D fingerprint templates.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation application of U.S. applicationhaving Ser. No. 13/935,584 filed on 5 Jul. 2013, which claims benefitunder 35 U.S.C. §119(e) of U.S. Provisional Application having Ser. No.61/680,716 filed 8 Aug. 2012, which is hereby incorporated by referenceherein in its entirety.

FIELD OF INVENTION

This invention relates to a method of 3D biometrics identification, inparticular the present invention is used to identify fingerprint.

BACKGROUND OF INVENTION

Biometrics based human identification is one of the most critical andchallenging task to meet growing demand for stringent security. Amongdifferent biometric features, fingerprint-based biometric is the mostproven technique and has the largest market shares. Although fingerprintrecognition has been studied for many years and much progress has beenmade, even the performance of state-of-the-art fingerprint matchers isstill much lower than the expectations of people and theory estimation.The reason is that the traditional fingerprint acquisition is conductedby pressing or rolling of fingers against the hard surface (glass,silicon, polymer) or paper which often results in partial or degradedimages due to improper finger placement, skin deformation, slippages,smearing or due to sensor noise. Therefore contactless finger imagingsystems that can provide three dimensional (3D) representation offingerprints have recently emerged to provide ideal solutions to aboveintrinsic problems. Such 3D approaches can provide more accuratepersonal identification due to the large amount of information containedin the 3D surface models as compared to the traditional two dimensional(2D) fingerprint images. The main obstacle of these emerging 3Dfingerprint technologies to replace the conventional 2D fingerprintsystem lies in their bulk and high cost, which mainly results from theusage of structured lighting system or multiple cameras.

Besides, both of these technologies have not been able to exploit othersurface parameters, like surface normal vectors, refraction parameters,and scattering parameters etc., which can also contribute to theprecision of the reconstruction result. Therefore advanced capability todevelop more and low-cost solution for the 3D fingerprint identificationwill significantly enhance the applicability of fingerprint recognitiontechnologies in wide range of civilian and commercial applications.

SUMMARY OF INVENTION

In the light of the foregoing background, it is an object of the presentinvention to provide an alternative design of contactless 3D biometricfeature identification system and the method thereof.

Accordingly, the present invention, in one aspect, is a method ofidentifying biometric feature comprising the steps of capturing aplurality of images with different illuminations of an object that hassuch biometric features; reconstructing a 3D surface model of the objectbased on the captured images and extracting a plurality of features fromthe plurality of images and the 3D surface model. The aforesaidplurality of features comprises 3D coordinates and orientations of thebiometric features

In one embodiment of the present invention, the plurality of featuresfurther comprises surface curvature and local surface orientations ofthe object and 2D coordinated and orientation of the biometric features.

In another embodiment, the step of extracting the plurality of featuresfurther comprises the steps of extracting 2D coordinates andorientations of the features from each of the captured images;evaluating the reliability of the features accordingly; and extracting3D coordinates and orientations of the features based on thereliability.

In a further embodiment, the step of establishing identity furthercomprises the steps of computing a plurality of parameters based on theplurality of features and corresponding predefined reference features ofa second object; generating matching scores based on the plurality ofparameters; and establishing identity of the biometric feature based onthe matching score. In one embodiment, a combined matching score whichis a weighted sum of said matching scores. In another embodiment, thecombined matching score is a dynamic combination of said matchingscores.

According to another aspect of the present invention, a biometricfeature identification system is disclosed. The system comprises alighting module configured produce a plurality of illuminations on theobject having the biometric features; a fix-viewpoint image capturingmeans configured to capture images of an object having the biometricfeatures; a microprocessor coupled to the image capturing means and thelighting module; and a computer-readable storage medium coupled to themicroprocessor. The computer-readable storage medium is encoded withcomputer-readable instructions for causing the microprocessor to executethe following steps: synchronizing the image capturing means and thelighting module; capturing a plurality of images of the object with eachimage having different illuminations; reconstructing a 3D surface modelof the object based on the plurality of images; extracting a pluralityof features from the plurality of images and the 3D surface model; andestablishing identity of the object accordingly.

In one embodiment, the plurality of features comprises 2D and 3Dcoordinates and orientation of the biometric features; as well assurface curvature and local surface orientations of the object.

In one embodiment of the present invention, the lighting modulecomprises a plurality of light sources positioned around the imagecapturing means which can be switched on and off independently by themicroprocessor thereby producing different illuminations on the object.

In another embodiment of the present invention, the lighting modulecomprises a modulated light source wherein the output intensity iscontrollable by the microprocessor thereby producing differentilluminations on the object.

According to another aspect of the present invention, acomputer-implemented method of identifying a 3D fingerprint in aspecially developed device comprising the steps of (i) acquiring 3Dimage(s) of fingerprint with the specially developed device; (ii)performing image normalization and feature extraction steps involving,noise elimination, ridge extraction, ridge modification and ridgevalidation; (iii) extracting 3D fingerprint minutiae in a 3D space andperforming the reliability evaluation of the extracted 3D fingerprintminutiae; (iv) automatically extracting of 3D fingerprint templatecomprising at least five features, including location of minutiae in a3D space using its x, y, z location, and its orientation in a 3D spacecomprising θ (azimuth angle) and φ (elevation angle); (v) automaticallyregistering two 3D fingerprint templates, say P and Q, by computingrelative 3D minutiae representation from a 3D minutiae in P (sample)considered as the origin and aligning every 3D minutiae in Q to computebest possible matching score, wherein every 3D minutiae in template P isfurther considered as the origin for the alignment of 3D minutiae intemplate Q, and the best possible matching score obtained from suchconsiderations (candidate alignments) is considered as the finalmatching score between two 3D fingerprint templates, wherein thematching score is generated between two 3D minutiae templates withmultiple features by counting the fraction of matched minutiae in two 3Dtemplates (P and Q) and the two 3D minutiae are considered as matched ifdifference between their feature(s) is smaller than a threshold(s) ortolerance limit(s), whereby the matching score between two unique 3Dfingerprint template computed between the presented fingerprint andthose stored in a registration database is used to establish theidentity of the fingerprint.

In one embodiment, the matching scores from the 3D fingerprint minutiaeare combined with the matching scores from the 3D fingerprint curvature,3D unit normal vector matching scores, and 2D fingerprint matchingscores, which are also simultaneously generated from the 3D fingerprintimages to form the combined matching scores.

There are many advantages to the present invention. Firstly, thisinvention can be used in a variety of biometric identificationapplications, including fingerprint identification. The presentinvention utilizes a single camera with fixed viewpoint thatsimultaneously acquires and processes multiple 2D fingerprint imagesunder different illuminations, thus reducing system cost substantially.Such photometric stereo inspired approach provides a low-costalternative to the currently available 3D fingerprint recognitionsystems which either use structured lighting or multiple cameras.

Another advantage of the present invention is that the accuracy offingerprint recognition is higher than conventional fingerprint systemsin both verification and recognition applications. This improvement isdue to the 3D feature extraction algorithm as proposed in the presentinvention which is able to recover additional fingerprint features andadaptively match them.

BRIEF DESCRIPTION OF FIGURES

FIG. 1 is a schematic diagram of biometric feature identification systemaccording to one of the embodiment of the present invention.

FIG. 2 is the front view of the image capturing means and the lightingmodule according to one of the embodiments of the present invention.

FIG. 3 shows the concept flow chart of the method of identifyingbiometric feature according to one of the embodiments of the presentinvention.

FIG. 4 shows the relative localization of a 3D minutia (m) with areference minutia (m_(r)).

FIG. 5 shows a plurality of images captured by the biometric featureidentification system according to one embodiment of the presentinvention.

FIG. 6 shows four different series of 2D fingerprint images and thereconstructed 3D surface model, in particular (a) images as obtained bythe image capturing mean according to one embodiment of the presentinvention; (b) images after normalization; (c) enhanced normalizedimages; (d) 3D curvature images of corresponding fingers and (e)reconstructed 3D surface model of the finger.

FIG. 7 shows the receiver operating characteristic (ROC) curves using 2Dfingerprint images with different illuminations.

FIGS. 8 a and 8 b shows the ROC curve for the system performance usingreconstructed 3D surface model and the system performance usingcombinations of 3D and 2D fingerprint images as acquired according toone embodiment of the present invention respectively.

FIG. 9 a shows the validation performance of the biometric featuresidentification system according to the same embodiment of the presentinvention using 240 known clients and 20 unknowns. FIG. 9 b shows thefalse positive identification rate (FPIR) vs. false negativeidentification rate (FNIR) characteristics from the same experiment asin FIG. 9 a.

FIGS. 10 a and 10 b shows the cumulative match characteristics (CMC) forthe average recognition performance on using reconstructed 3D surfacemodel and respective performance using combination of 3D model and 2Dfingerprint images according to the same embodiment of the presentinvention.

FIG. 11 a and 11 b shows ROC curve for the system performance usingdifferent surface code matching scores and respective performance usingcombination of different surface code and local surface orientationaccording to the same embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

As used herein and in the claims, “comprising” means including thefollowing elements but not excluding others.

Referring now to FIG. 1, the first aspect of the present invention is abiometric feature identification system 20 comprising a lighting module22, an image capturing means 24 and a microcomputer 26. Themicrocomputer 26 at least comprises a microprocessor (not shown) and acomputer-readable storage medium or memory (not shown) connected to themicroprocessor. The lighting module 22 and image capturing means 24 arefurther coupled to the microprocessor.

The lighting module 22 is configured to produce a plurality ofilluminations on an object having the biometric features and the imagecapturing means 24 is configured to capture a plurality of images of theobject 18 having the biometric features of interest. During operation,the object 18 has no physical contact with the biometric featureidentification system 20. In one embodiment, the object 18 is placed infront of, without any physical contact with, the image capturing means24 as shown in FIG. 1. As shown in this figure, the object 18 is afinger and the biometric features are the fingerprint.

Referring now to FIG. 2, in one embodiment of the present invention, thelighting module 22 comprises of a plurality of light-emitting diodes 28a-28 g (LEDs). The LEDs 28 a-28 g are positioned around the imagecapturing means 24 and are distributed evenly around the center of theimage capturing means 24. The LEDs 28 a-28 g are configured to beswitched on or off independently by the microprocessor thereby theobject can be illuminated from different illuminating angles. Themicroprocessor is further configured to synchronize the lighting module22 and image capturing means 24 such that a plurality images can becaptured with each of the images having different illuminations on thesame object. In another embodiment, the illumination sources that emitdifferent colored light can be employed to simultaneously estimate theshading corresponding to each color source by separating thecolor/spectral image components of acquired/multispectral image(s). Inanother embodiment, the plurality of LEDs can be replaced by a singlemodulated light source and different illuminations are produced on theobject by alternating the output intensity of the modulated lightsource.

According to another aspect of the present invention, now referring toFIG. 3, a method of identifying biometric feature is provided. Aftercapturing a plurality of images of the object in step 30, a 3D surfacemodel of the object is reconstructed in step 32. Afterwards a pluralityof features is extracted from the images and 3D surface model in step 34which in turns are used in establishing the identity of the biometricfeatures (step 36).

The reconstruction step 32 is based on the shape from shading technique.Given a captured image E(x, y), the shape from shading technique can beused to recover the surface model z(x, y) using the following equation(1):

E(x,y)=ρl ₀ R(p(x,y),q(x,y))   (1)

where ρ is the albedo and I₀ is the incident radiance. The surfacereflectance R relates the observed image pixel intensities in E(x, y) tothe surface orientations/gradients for a given source direction and thesurface reflectance in the direction of the image capturing means. Thesurface gradients p(x, y) and q(x, y) can be defined as followingequation (2):

p(x,y)=∂ƒ(x,y)/∂x, q(x,y)=∂ƒ(x,y)/∂y   (2)

The 3D surface model can be reconstructed by recovering the surfaceheight information z=ƒ(x, y). In one embodiment, the surface of theobject is assumed to be a Lambertian surface which is illuminated by aplurality of, say m, light sources, i.e., L=[l¹,l², . . . l^(m)]^(T).Each of these light sources is fixed with known directionl=[l_(x),l_(y),l_(z)]^(T) and radiance ∥l∥. Letn=[n_(x),n_(y),n_(z)]^(T) be the unit surface normal vectors at somepoint of interest on the surface model. The observed image pixelintensities y, from the m images, corresponding to the respectiveillumination sources can be written as following equation (3):

y=L.x   (3)

where L=[y₁,y₂, . . . y_(m)]^(T) and x=ρ[n_(x),n_(y),n_(z)]^(T). In oneembodiment, the light source directions are not co-planer so that thematrix L is non-singular. Equation (3) illustrates linear relationshipbetween the 3D surface model, observed pixel intensities from thecaptured images and the unit surface normal vectors x. The unknownvector x can be estimated from the least squared error technique usingthe following equation (4):

x=(L ^(T) L)⁻¹ L ^(T) y≡ρn   (4)

Since n is of unit length, the length of recovered vector x is theabsolute reflectance (albedo) ρ and the surface normal is represented bythe direction of unit vector n. The recovered surface normals are thenintegrated to recover the 3D object surface z(x, y).

In one embodiment, the method of identifying biometric feature furthercomprises a step of preprocessing the 3D surface model z(x, y). The 3Dsurface model is first smoothed using a median filer, followed by a 3Dvariant of Laplacian smoothing which can effectively denoises pointclouds. For a vertex z_(t) within the image z=ƒ(x, y), with itsneighbors z_(j), the updated vertexes z _(t) are computed by thefollowing equation (5):

$\begin{matrix}{{\overset{\_}{z}}_{\iota} = {{\left( {1 - \varepsilon} \right)z_{i}} + {\frac{\varepsilon}{\sum_{j}w_{ij}}{\sum_{j}{w_{ij}z_{j}}}}}} & (5)\end{matrix}$

where w_(ij) is a finite support weighting function and is chosen as theinverse of distance between vertex z_(t) and its neighbors z_(j), i.e.w_(tj)=∥z_(j)−z_(i)∥⁻¹. The reconstructed surface model is smoothedafter 40 iterations with ∈=0.5 and the neighbors j are chosen within ±2pixel in the x and y directions from vertex z_(t). The normal vectors ofthe cloud point data for the smoothed surface is then computed by thegradient of z=ƒ(x, y). The normalized vector is an upward normal with(−g_(x), −g_(y), 1), where g_(x) and g_(y) are the gradient along x andy directions. The normalized surface normal is then used for estimatingthe principle curvature.

After reconstructing the 3D surface model in step 32, a plurality offeatures is extracted from the 3D surface model and the plurality ofimages in step 34. In one embodiment, the features comprise 3Dcoordinates and orientations of the biometric features. In anotherembodiment of the present invention, the features further comprisesurface curvature and local surface orientation of the object. In orderto prevent object movement between each captured image from affectingthe accuracy and reliability of the extracted features, in oneembodiment, the 3D coordinates and orientations are extracted base onthe result of a reliability test applied on the plurality of images.

To illustrate the feature extraction algorithm of the present invention,a specific realization on applying these extraction algorithms toidentifying the fingerprint is disclosed herein. In the context offingerprint identification, the aforesaid object is a finger and thebiometric feature is the fingerprint itself. In one embodiment, the stepof extraction 34 starts with extracting 2D coordinates and orientationsof the features (2D minutiae). 2D minutiae details (x, y, θ, q)consisting of position of the minutiae (x, y), angle θ representing theorientation of the minutiae and the quality q of the minutiae.

In one embodiment, the reliabilities of the extracted 2D minutiae arethen evaluated. The rationale behind the reliability test is that if aparticular minutiae is detected in more than one image among theplurality of images acquired, that particular minutiae is considered tobe reliable. One particular implementation of the reliability test isdisclosed as follows:

Let the list of minutiae extracted from the plurality of fingerprintimage under the first illumination angle be L={m₁, m₂ . . . m_(n)} wherem=[x, y, θ, q]. Then the counted list CL={cm₁, cm₂, . . . cm_(n)} isinitialized, where cm=[x, y, θ, q, c], and c is the number ofoccurrences and is set to 1. For each minutiae m_(i) from thefingerprint image under the second to seventh illumination angle, the CLis updated as follows:

Let {TL} be a subset of CL such that such that ∥x_(cm) _(k)−x_(i),y_(cm) _(k) −y_(i)∥²≦k₁ and min(|θ_(cm) _(k) −θ_(i)|,360−|θ_(cm)_(k) −θ_(i)|)≦k₂, where cm_(k)∈CL, then cm_(t) is updated such thatc_(t)≧c_(i) for all cm_(i)∈{TL}. The value of x, y, θ, q of updatedcm_(t) will be the average value of existing cluster members and the newmember, and c will be increased by one. If {TL}=Ø, then CL=CL ∪[x_(i),y_(i),θ_(i),q_(i),1]. In one embodiment of the present invention,the constant k₁ is set to 4 since the minutiae location in differentlyilluminated fingerprint images would not shift too far away and squareroot of 4, i.e. 2, which is slightly smaller the half width of the ridge(measured as 5 pixels in experiments). In another embodiment, constantk₂ is set to 25 for the acceptable angle difference which can keep thecluster to have similar direction/orientation. After updating CL, thesubset of CL is picked as DL with c≧2. If two clusters groups are tooclose, they are merged to one single group to reduce the chance that asingle minutia is extracted as two minutiae. The final list of minutia,which is the merged list DL, is considered to have high reliability.

In one embodiment, the highly reliable 2D minutiae are extended to 3Dcoordinates and orientations (3D minutiae) by adding two additionalfeatures z and φ onto (x, y, θ, q). The value z is the height of thevertex on the reconstructed 3D surface model at position (x, y) while θand φ represent the minutiae orientation in spherical coordinates withunit length 1. The angle φ is computed by tracing the reconstructed 3Dsurface model at minutiae locations along the direction of θ, which isavailable from 2D minutiae details (x, y, θ, q). In case of bifurcationtype of minutiae, the local ridge surface is masked while local valleysurface is masked for the end type of minutiae since the direction θ ishere pointing in outward direction. The angle φ is then computed byestimating the principle axes of the masked ridge surface. Therefore,the minutiae representation can be extended in 3D space with therepresentation (x, y, z, θ, φ, q).

In one embodiment, surface curvature and surface directions are alsoextracted as members of the plurality of features. The surface curvaturetypically measures local bending of fingerprint surface at each of thesurface points while the surface directions indicate the directions ofminimum and maximum surface bending. The surface curvature is estimatedusing local surface fitting (cubic order approximation). Let a givensurface point be s with its normal N and its u neighboring points be t,with their normal vectors K_(i) where i=1, 2 . . . u. In the coordinatesystem with s as the origin (0, 0, 0) and its normal N as the z-axis,the position of neighbors t_(i) is (x_(i), y_(i), z_(i)) and the valuesof K_(i) is (a_(i), b_(i), c_(i)). Using the adjacent-normal cubic orderalgorithm, a surface that can fit the vertex and its neighboring pointsis located such that:

$\begin{matrix}{z = {{f\left( {x,y} \right)} = {{\frac{a}{2}x^{2}} + {bxy} + {\frac{c}{2}y^{2}} + {dx}^{3} + {{ex}^{2}y} + {fxy}^{2} + {gy}^{3}}}} & (6)\end{matrix}$

The normal vector of the surface point s in the approximated surface iswritten as

$\begin{matrix}\begin{matrix}{{N\left( {x,y} \right)} = \left( {{f_{x}\left( {x,y} \right)},{f_{y}\left( {x,y} \right)},{- 1}} \right)} \\{= \left( {{{ax} + {by} + {3\; {dx}^{2}} + {exy} + {fy}^{2}},} \right.} \\\left. {{{bx} + {cy} + {{ex}\; 2} + {2\; {fxy}} + {3\; {gy}^{2}}},{- 1}} \right)\end{matrix} & (7)\end{matrix}$

The cubic-order surface fitting, for both the neighboring surface pointsand their normal, generate following three equations for each of thesurface points.

$\begin{matrix}{{\begin{pmatrix}{0.5\; x_{i}^{2}} & {x_{i}y_{i}} & {0.5\; y_{i}^{2}} & x_{i}^{3} & {x_{i}^{2}y_{i}} & {x_{i}y_{i}^{2}} & y_{i}^{3} \\x_{i} & y_{i} & 0 & {3\; x_{i}^{2}} & {2\; x_{i}y_{i}} & y_{i}^{2} & 0 \\0 & x_{i} & y_{i} & 0 & x_{i}^{2} & {2\; x_{i}y_{i}} & {3\; y_{i}^{2}}\end{pmatrix}\Omega} = \begin{pmatrix}z_{i} \\{{- a_{i}}/c_{i}} \\{{- b_{i}}/c_{i}}\end{pmatrix}} & (8)\end{matrix}$

where Ω=[a b c d e f g]^(T) is the coefficient vector of cubic surface.Equation (7) represents an over determined equation system and can bewritten in the following form:

KΩ=R   (9)

where K is 3u×7 matrix (from left hand side of equation 7) and R is 3u×1vector. Least-square fitting is applied to find the best solution forequation (7) and construct Weingarten curvature matrix W for the fittedsurface using only three coefficients.

$\begin{matrix}{W = \begin{pmatrix}a & b \\b & c\end{pmatrix}} & (10)\end{matrix}$

The eigenvalues of Weingarten matrix are the maximum and minimumprinciple curvature of the surface (k_(max) and k_(min)), and theireigenvectors are the principal direction vectors and (t_(max) andt_(min)) which can be directly computed. Shape index of a surface atvertex s is adopted to quantify the local shape of the fingerprintsurface. The curvature information described using C_(i)(s) isindependent of scale and can be computed using following equation (11):

$\begin{matrix}{{C_{i}(s)} = {\frac{1}{2} - {\left( \frac{1}{n} \right){\tan^{- 1}\left( \frac{t_{\max} + t_{\min}}{t_{\max} - t_{\min}} \right)}}}} & (11)\end{matrix}$

The C_(i)(s) maps all shapes in the interval [0,1]. In one embodiment,the shape of the surface is considered to be valley and ridge ifC_(i)(s) is close to 0.25 and 0.75 respectively. This binomialrepresentation is referred as Surface Code. In another embodiment, thezone of surface is split into five zones: cup, rut, saddle, ridge andcap as shown in table 1.

TABLE 1 Zones of Finger Surface code Shape index 0-0.0625 0.0625-0.43750.4375-0.5625 0.5625-0.9375 0.9375-1 Angle (π/6) / 0 1 2 3 4 5 / 0 1 2 34 5 / Code 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

Direction of the dominant principle curvature (max(|k_(Max)|,|k_(Min)|)is portioned into six directions. Rut and ridge zones are furtherdivided since cup, saddle and cap's |k_(Max)| and |k_(Min)| are close;therefore, t_(max) and t_(min) are not as accurate as that in rut andridge zones. The resulting feature representation, referred to as FingerSurface Code has 15 different values and therefore 4-bits can storeresulting binary code for each pixel.

In one embodiment, local surface orientation is also extracted from thereconstructed 3D surface model. The local surface orientation isextracted by exploiting the unit normal vector location at the 3Dfingerprint surface points. The unit normal vector at 3D fingerprintsurface point can be estimated using lowest eigenvalues which areavailable while determining principle axes of the masked ridge surfacefor the minutiae direction angle φ. Since the direction of principleaxis is the normal vector over the masked ridge surface, it has leastnoise as compared to the case when the normal vector is measured on theexact minutiae location.

After extracting all or part of the features as mentioned above, theidentity of the biometric features is established based on the extractedfeatures. In one embodiment, step 36 of establishing the identityfurther comprises the steps of computing a plurality of parameters basedon the extracted features and corresponding predefined reference valuesof a second object; generating a matching score based on the parameters;and establishing identity of the object based on the matching score. Inone embodiment, reference values of the second object corresponds to thebiometric features obtained using the same feature extraction process onthat object and stored in a database.

In one embodiment, the matching score is computed merely based on the 2Dminutiae, and is referred as 2D matching score. The extracted minutiaepairs from the query image (Q) and pre-obtained template image (P) arematched to generate a 2D fingerprint matching scores between two 2Dfingerprint images. All minutiae are converted to spherical coordinateas [r, A_(s), A_(θ), T] which center at the reference minutia and alignthe angle with θ_(ref).

$\begin{matrix}{r = \sqrt{\left( {x - x_{r}} \right)^{2} + \left( {y - y_{r}} \right)^{2}}} & (12) \\{A_{s} = {{{atan}\; 2\left( \frac{y - y_{r}}{x - x_{r}} \right)} - \theta_{r}}} & (13) \\{A_{\theta} = \left( {\theta - \theta_{r}} \right)} & (14)\end{matrix}$

where r is the distance of respective minutiae with the referenceminutia, A_(s) is the angular separation of the minutia and A_(θ) is theorientation of the minutia. If the difference between [r_(i), A_(si),A_(θi), T_(i)] in P and [r_(j), A_(sj), A_(θj), T_(j)] in Q is smallerthan a predetermined threshold, then T_(i)=T_(j), and the minutia i in Pand minutia j in Q are considered as matched pair. The overall 2Dmatching score S_(2D) is generated using the equation (15):

$\begin{matrix}{S_{2\; D} = \frac{m^{2}}{M_{p}M_{Q}}} & (15)\end{matrix}$

where m is the total number of matched minutiae pairs and M_(P), M_(Q)is the number of minutiae in query and template image respectively. Themaximum score from all of the possible reference minutia pair isselected as the final 2D matching score between fingerprint image P andQ.

In another embodiment, the matching score is computed merely based onthe 3D minutiae, and is referred as 3D matching score. Similar to the 2Dminutiae matching score, a minutiae pair from the query image (Q) andpre-obtained template image (P) is first selected and all minutiae areconverted to spherical coordinate. The minutiae pair are then alignedwith the x-axes and z-axes. This alignment ensures that the alignedminutiae (in both P and Q) location can serve as the universal origin orreference minutiae to measure other minutiae in respective templates. Analigned minutia represented as m_(r)=[x_(r), y_(r), z_(r), θ_(r), φ_(r)]in template P, the relative representation of other 3D minutiae intemplate P can be denoted as m=[r, A_(s), A_(θ), A_(g), A_(φ)].Referring to FIG. 4, r is the radial distance with reference minutiae,A_(θ), is the azimuth angle and A_(φ) is the elevation angle thatlocalize the minutiae m in 3D plane, while A_(s) and A_(g) are theazimuth and the elevation angle that localize the radial vector r (withrespect to reference minutiae m_(r) in) 3D space. Let R_(z)(θ) andR_(y)(φ) be the rotation matrix along z and y direction in Cartesiancoordinate, and sph(x, y, z) be the Cartesian to Spherical coordinatetransformation with unit length 1:

$\begin{matrix}{{{R_{z}(\theta)} = \begin{bmatrix}{\cos \; \theta} & {{- \sin}\; \theta} & 0 \\{\sin \; \theta} & {\cos \; \theta} & 0 \\0 & 0 & 1\end{bmatrix}},{{R_{y}(\varphi)} = \begin{bmatrix}{\cos \; \varphi} & 0 & {{- \sin}\; \varphi} \\0 & 1 & 0 \\{\sin \; \varphi} & 0 & {\cos \; \varphi}\end{bmatrix}}} & (16) \\{{{sph}\left( \lbrack{xyz}\rbrack \right)} = \begin{bmatrix}{{atan}\; 2\left( {y,x} \right)} & {\sin^{- 1}z}\end{bmatrix}} & (17)\end{matrix}$

where atan2 is the four-quadrant inverse tangent function. Theparameters for the relative representation (feature vector) of minutiaem are computed as follows:

$\begin{matrix}{r = \sqrt{\left( {x - x_{r}} \right)^{2} + \left( {y - y_{r}} \right)^{2} + \left( {z - z_{r}} \right)^{2}}} & (18) \\{\left\lbrack {x^{\prime}y^{\prime}z^{\prime}} \right\rbrack = {{R_{y}\left( {- \varphi_{r}} \right)}{R_{z}\left( {- \varphi_{r}} \right)}{\frac{1}{r}\begin{bmatrix}{x - x_{r}} & {y - y_{r}} & {z - z_{r}}\end{bmatrix}}^{T}}} & (19) \\{\left\lbrack {A_{s}A_{g}} \right\rbrack = {{sph}\left( \left\lbrack {x^{\prime}y^{\prime}z^{\prime}} \right\rbrack \right)}} & (20) \\{\left\lbrack {A_{\theta}A_{\varphi}} \right\rbrack = {{sph}\left( \left( {{R_{y}\left( {- \varphi_{r}} \right)}{R_{z}\left( {- \varphi_{r}} \right)}\left( {{sph}^{- 1}\left( \left\lbrack {\theta \mspace{14mu} \varphi} \right\rbrack \right)} \right)^{T}} \right)^{T} \right)}} & (21)\end{matrix}$

Two 3D minutiae in the two fingerprint template P and Q are consideredas matched pair if the difference between their feature vectors

(r_(P_(i)), A_(s_(P_(i))), A_(θ_(P_(i))), A_(g_(P_(i))), A_(φ_(P_(i))))  and  (r_(Q_(i)), A_(s_(Q_(i))), A_(θ_(Q_(i))), A_(g_(Q_(i))), A_(φ_(Q_(i))))

is smaller than a given threshold. The differences of each elements inthe feature vector are computed as follows:

$\begin{matrix}{{\Delta \; r} = {{r_{P_{i}} - r_{Q_{i}}}}} & (22) \\{{\Delta \; A_{s}} = {\min \left( {{{A_{s_{P_{i}}} - A_{s_{Q_{i}}}}},{{360{^\circ}} - {{A_{s_{P_{i}}} - A_{s_{Q_{i}}}}}}} \right)}} & (23) \\{{\Delta \; A_{\theta}} = {\min \left( {{{A_{\theta_{P_{i}}} - A_{\theta_{Q_{i}}}}},{{360{^\circ}} - {{A_{\theta_{P_{i}}} - A_{\theta_{Q_{i}}}}}}} \right)}} & (24) \\{{\Delta \; A_{g}} = {{A_{g_{P_{i}}} - A_{g_{Q_{i}}}}}} & (25) \\{{\Delta \; A_{\varphi}} = {{A_{\varphi_{P_{i}}} - A_{\varphi_{Q_{i}}}}}} & (26)\end{matrix}$

In one embodiment of the present invention, the pair of minutia isconsidered as a match pair if all the difference vectors (22)-(26) aresmaller than their corresponding predefined thresholds.

In another embodiment of the present invention, a unified 3D matchingdistance between two 3D minutiae is generated and rejects falselymatched minutiae using comparison with a predefined threshold. Thefunction to combine the difference vector in equations (22)-(26) forgenerating unified matching distance can be written as follows equation(27):

$\begin{matrix}{{{funRSG}\left( {\Delta \; v} \right)} = {\left( \frac{\Delta \; r}{65} \right)^{0.8} + \left( \frac{\Delta \; A_{s}}{30} \right)^{0.8} + \left( \frac{\Delta \; A_{g}}{15} \right)^{0.8} + \left( \frac{\Delta \; A_{\theta}}{18} \right)^{0.8} + \left( \frac{\Delta \; A_{\varphi}}{42} \right)^{0.8} + \left( \frac{1 - {\cos \; N}}{0.075} \right)^{0.8}}} & (27)\end{matrix}$

where Δν are the vector of difference of values as computed in(22)-(26). In one embodiment, the pair of minutia is considered as amatched pair if funRSG(Δν) is smaller than 1.825.

In another embodiment, the equation (27) can be generalized and improvedto take into the consideration of the non-linearity of the differencevector in equations (22)-(26) by adding the function of distance fromreference minutiae to obtained equation (28):

$\begin{matrix}{{{funRSG}\left( {\Delta \; v} \right)} = {\left( \frac{\Delta \; r}{f(r)} \right)^{a} + \left( \frac{\Delta \; A_{s}}{A} \right)^{b} + \left( \frac{\Delta \; A_{g}}{B} \right)^{c} + \left( \frac{\Delta \; A_{\theta}}{C} \right)^{d} + \left( \frac{\Delta \; A_{\varphi}}{f(r)} \right)^{0.8} + \left( \frac{1 - {\cos \; N}}{D} \right)^{f}}} & (28)\end{matrix}$

which has an independent set of power term {a, b, . . . ,f} and f(r) isthe function of distance. The pair of minutia is considered as a matchedpair if funRSG(Δν) is smaller than a predefined threshold.

The overall 3D matching score S_(3D) between two 3D minutiae template Pand Q is computed as follows:

$\begin{matrix}{S_{3\; D} = \frac{m^{2}}{M_{p}M_{q}}} & (29)\end{matrix}$

where m is here refers to the total number of 3D matched minutiae pairsand M_(P), M_(Q) is the number of 3D minutiae in query and templateimage respectively.

In yet another embodiment, the matching score is computed merely basedon the Finger Surface Codes. The Finger Surface Codes matching scoreS_(Finger Surface Code) between a query Finger Surface Codes U from thequery image and a reference Finger Surface Codes V from pre-obtainedtemplate image is computed using their normalized hamming distance asshown in equation (30):

$\begin{matrix}{S_{{Finger}\mspace{11mu} {Surface}\mspace{14mu} {Code}} = {\frac{1}{4 \times U \times V}{\sum\limits_{p = 1}^{U}{\sum\limits_{q = 1}^{V}{\otimes \left( {{J\left( {p,q} \right)},{K\left( {p,q} \right)}} \right)}}}}} & (30)\end{matrix}$

where {circle around (x)} denotes the Hamming distance between the twofour bit Finger Surface Codes.

In another embodiment, the matching score is computed merely based onthe local surface orientation (i.e. unit normal vector). Thesurface-orientation matching score S_(orientation) between two unitnormal vectors, say n_(p) and n_(q) from the minutiae of the query image(Q) and pre-obtained template image (P), is generated using their dotproduct, i.e., S_(orientation)=cos α=n_(p)√n_(q). If S_(orientation) isbigger than a pre-defined threshold, the 430 normal vectors of the two3D fingerprints are considered to be matched.

In one embodiment, a combined matching score is computed and is used forminutiae matching. In a further embodiment, the combined matching scoreis a weighted sum of S_(2D), S_(3D), S_(Finger Surface Codes) andS_(orientation). In another embodiment, the combined matching score is adynamic combination of S_(2D), S_(3D), S_(orientation) andS_(Finger Surface Codes).

In order to demonstrate the flexibility of the present invention, anexperiment with real data is conducted. A 3D fingerprint database with240 distinct fingers using the biometric feature system as proposed isfirst obtained. The light module comprises seven different LEDs whichare able to generate seven different illuminations on the same fingerand seven images (impressions) are acquired for each of the fingers asshown in FIG. 5.

FIG. 6 illustrates image samples from four different fingers (014_r3_g5,015_I3_g4, 003_I4_g5 and 021_I2_g3) and the intermediate results duringpreprocessing and 3D reconstruction. FIG. 6 a shows one of the imagescaptured by the image capturing means for each finger. FIGS. 6 b and 6 cillustrate the images after normalization and enhancement respectively.FIGS. 6 d and 6 e shows the surface curvature images and thereconstructed 3D surface model.

As mentioned, multiple 2D images acquired for the 3D fingerprintreconstruction can themselves be utilized for generating 2D matchingscores based on the 2D minutiae. However the imaging procedure employedin the present invention requires that each of these images be acquiredunder different illumination. Therefore the 2D minutiae extracted fromthe respective images can be different and optimal matching scores aregenerated by matching all the available images from two fingers intemplate P and Q. FIG. 7 illustrates experimental results from the usageof all 2D fingerprint images when (i) the best possible matching scoresamong matching all seven images from each fingers are used and when (ii)only best performing one image was used. It should be noted that the 2Dmatching score generated using all respective 2D fingerprint imagesoutperforms that utilizing only the best performing image.

The performances of different fingerprint matching scores as proposed inthe present invention are also studied. FIG. 8 a illustrates matchingresults using 3D minutiae representation, 3D fingerprint curvaturerepresentation and also using 2D fingerprint images acquired for the 3Dfingerprint reconstruction. It can be ascertained from this figure that3D minutiae representation achieves superior results as compared tothose using 3D fingerprint surface representation. FIG. 8 b illustratesexperimental results from the combination of 2D matching score (usingthe images acquired during 3D fingerprint reconstruction), 3D matchingscore, and 3D surface-orientation matching score. It can be ascertainedfrom this figure that the combination of 3D minutiae matching scoreswith the available 2D minutiae matching scores achieves superiorperformance This performance improvement is significant and suggeststhat 3D fingerprint information generated according to the presentinvention can be effectively used to improve performance for thecontactless 2D fingerprint matching.

In order to ascertain the capability of the present invention toidentify the unknown clients, 20 new finger images are acquired and areemployed as unknown clients. These unknown clients were then identifiedfrom the proposed method to ascertain the performance. FIG. 9 a showsthe plot of number of unknown clients identified as unknown verses knownclients rejected as unknown. These experimental results suggest superiorperformance from 3D minutiae representation and illustrate performanceimprovement with the combination of conventional 2D minutiae features.The performance from the proposed identification method for the FPIR andFNIR was also observed and is illustrated in FIG. 9 b. The performanceimprovement using the combination of 3D minutiae representation,extracted from the reconstructed 3D fingerprint images, and 2D minutiaerepresentation is observed to be quite consistent in FPIR vs. FNIRplots.

The present invention is also configured to perform recognition tasks ofbiometric features. FIGS. 10 a and 10 b illustrate the CMC from therecognition experiments for the comparison and combination of 2D/3Dfingerprint features. These results also suggest superior performanceusing 3D minutiae representation, over 3D curvature representation, andalso illustrate improvement in average (rank-one) recognition accuracyusing the combination of available minutiae features from the 2Dfingerprint images acquired during the 3D fingerprint reconstruction.The superiority of finger surface code, over the surface code, can beascertained from the experimental results in FIG. 11 a. Robust 3Dfingerprint matching using the unit normal vector field matching and thesignificant performance improvement from the adaptive combination withfinger surface code based 3D curvature matching can be ascertained fromexperimental results in FIG. 11 b.

The overall quantitative performance comparison between conventionalidentification and different embodiments of the present invention isshown in the following table 2:

TABLE 2 Individual and combined match performance from 2D and 3DFingerprint Images 2D Minutiae + 2D Minutiae + Experiments 2D Minutiae3D Curvature 3D Minutia 3D Curvature 3D Minutiae Equal Error Rate 2.12%12.19% 2.73% 1.73% 1.02% from 240 clients Equal Error Rate 5.44% 32.26%9.28% 4.36% 3.33% from 240 clients and 20 unknowns Rank-1 accuracy94.56% 68.21% 90.72% 95.64% 96.67% from 240 clients and 20 unknowns

It should be ascertained from table 1 that the combination of 3Dminutiae matching scores with 2D minutiae matching scores achieves thebest performance. It suggests that the present invention provides a moreaccuracy fingerprint identification means comparing to the conventionalfingerprint systems.

The exemplary embodiments of the present invention are thus fullydescribed. Although the description referred to particular embodiments,it will be clear to one skilled in the art that the present inventionmay be practiced with variation of these specific details. Hence thisinvention should not be construed as limited to the embodiments setforth herein.

For instance, fingerprint identification is illustrated as a specificrealization of the method as proposed in the present invention. However,it should be clear that the present invention can be applied toidentification of any biometric features, for instance palm print,footprint, ear biometric and lip print.

Seven LEDs are used as the lighting module in the embodiment describedabove. It should be clear that any number of light sources may be usedaccording to the user's preference. In another embodiment, any number oflight sources with different color illuminations can be used, byseparating the spectral component for respective light source in theacquired multispectral image, to simultaneously determine shading fromeach of the colored light source. Moreover, any types of light sourcesmay also be used, such as light bulbs and fluorescent lamps, as long asthey are able to illuminate the object with biometric feature fromdifferent illumination angles.

What is claimed is:
 1. A computer-implemented method of identifying a 3Dfingerprint in a specially developed device comprising the steps of: (i)acquiring 3D image(s) of fingerprint with the specially developeddevice; (ii) performing image normalization and feature extraction stepsinvolving, noise elimination, ridge extraction, ridge modification andridge validation; (iii) extracting 3D fingerprint minutiae in a 3D spaceand performing the reliability evaluation of the extracted 3Dfingerprint minutiae; (iv) automatically extracting of 3D fingerprinttemplate comprising at least five features, including location ofminutiae in a 3D space using its x, y, z location, and its orientationin a 3D space comprising θ(azimuth angle) and φ(elevation angle); (v)automatically registering two 3D fingerprint templates, say P and Q, bycomputing relative 3D minutiae representation from a 3D minutiae in P(sample) considered as the origin and aligning every 3D minutiae in Q tocompute best possible matching score, wherein every 3D minutiae intemplate P is further considered as the origin for the alignment of 3Dminutiae in template Q, and the best possible matching score obtainedfrom such considerations (candidate alignments) is considered as thefinal matching score between two 3D fingerprint templates, wherein thematching score is generated between two 3D minutiae templates withmultiple features by counting the fraction of matched minutiae in two 3Dtemplates (P and Q) and the two 3D minutiae are considered as matched ifdifference between their feature(s) is smaller than a threshold(s) ortolerance limit(s), whereby the matching score between two unique 3Dfingerprint template computed between the presented fingerprint andthose stored in a registration database is used to establish theidentity of the fingerprint.
 2. The method according to claim 1 whereinthe matching score between two 3D minutiae is generated as an unifiedmatching score from all computed features using their nonlinearcombination for the specially developed 3D fingerprint device.
 3. Themethod according to claim 1 wherein the matching scores from the 3Dfingerprint minutiae are further integrated using the matching scoresfrom 3D fingerprint curvature matching scores which are alsosimultaneously generated from the acquired 3D fingerprint images.
 4. Themethod according to claim 1 wherein the matching scores from the 3Dfingerprint minutiae are further integrated using the matching scoresfrom the 3D unit normal vectors (representing local 3D surfaceorientations) matching scores which are also simultaneously generatedfrom the 3D fingerprint images.
 5. The method according to claim 1wherein the matching scores from the 3D fingerprint minutiae arecombined with the matching scores from the 3D fingerprint curvature, 3Dunit normal vector matching scores, and 2D fingerprint matching scores,which are also simultaneously generated from the 3D fingerprint imagesto form the combined matching scores.
 6. The method according to claim 5where the combined matching score is computed as the weighted sum ofindividual matching scores from the 2D fingerprint minutiae and thementioned 3D fingerprint minutiae.
 7. The method according to claim 5when the combined matching score is generated as the dynamic combinationof individual matching scores from the 2D fingerprint minutiae and thementioned 3D fingerprint minutiae.
 8. The method according to claim 1wherein the specially developed device is a mobile phone and thedifferent illuminations are provided by a single or multipleillumination sources like LED(s) or modulated built-in/camera flash of amobile phone.